Supplement to “ A Generalized Least Squares Matrix Decomposition ”

In addition to sparseness, there is much interest in penalties that encourage smoothness, especially in the context of functional data analysis. We show how the GPMF can be used with smooth penalties and propose a generalized gradient descent method to solve for these smooth GPMF factors. Many have proposed to obtain smoothness in the factors by using a quadratic penalty. Rice and Silverman (1991) suggested P (v) = v Ωv, where Ω is the matrix of squared second or fourth differences. As this penalty is not homogeneous of order one, we use the Ωnorm penalty: P (v) = (v Ωv)−1/2 = ||v ||Ω. Since this penalty is a norm or a semi-norm, the GPMF solution given in Theorem 2 can be employed. We seek to minimize the following Ω-norm penalized regression problem: 1 2 ||X Qu−v ||R + λv ||v ||Ω. Notice that this problem is similar in structure to the group lasso problem of Yuan and Lin (2006) with one group. To solve the Ω-norm penalized regression problem, we use a generalized gradient descent method. (We note that there are more elaborate first order solvers, introduced in recent works such as Becker et al. (2010, 2009), and we describe a simple version of such solvers). Suppose